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METHOD FOR STOPPING A FEIAGMENTATION OF DIGITAL IMAGES 



Field of the invention 

The present invention relates to a method for stopping 
the fragmentation of digital images, in parricular within the 
field of video image compression. 



Background of the invention 

To transmit or record digital images, use is commonly 
made of coding methods which reduce the quantity of information 
transmitted and, consequently, the bandwidth required for this 
transmission. 

Some of these coding methods call upon segmentation of 
the images into so-called homogeneous regions, that is to say 
into regions of uniform character according to one or more 
criteria such as the chrominance and/or the luminance of the 
pixels (image elements) of this region. 

Subsequently, the transmission of the luminance and 
chrominance data relating to each pixel of a region can be 
replaced by a simple transmission of the I'omdnance and 
chrominance data of the relevant region. 

Moreover, this segmentation can be callea upon m 
respect of inter-image coding. Thus, before transmitting or 
recording the information relating to the state, for example of 
luminance and chrominance, of a region thus defined, one 
determines whether this information has already been transmitted 



or recorded. Stated otherwise, one determines whether this 
region has already been identified in a previously transmitted 
image . 

If this region has not been identified in an earlier 
image, all the information relating to this region of the image 
is transmitted. 

Conversely, if this region has been identified during 
the transmission of a previous image, only a signal representing 
the displacement of this region with respect to the earlier 
image is transmitted. 

Thus, on reception or on reading, this region is 
reconstructed from the information already transmitted in 
respect of a previous image. 

For example, consider a series of images all 
comprising one and the same uniform and stationary blue 
background. In this case, by considering this blue background to 
form a single region, the information relating to this region is 
transmitted only for the first image. Thereafter, the coding of 
the following images only comprises indications indicating the 
presence and the location of this region. 

Thus, the blue background is recreated for all the 
images from the information transmitted (or recorded) in respect 
of the initial image. 

A description of a known method of segmenting im.ages 
is given nereinbelow with the aid of the appended figure. 

Represented in this figure are the various steps of a 
method 10 for the segmentation of images, this segm.entation 
comprising two preliminary operations, one, 20, relating to the 
luminance and the other, 22, relating to the chrominance or to 
analogous parameters as described later. 

It should be stressed that the operation 22 relating 
to the chrominance of the pixels is dubbed "fragmentation" while 
the final partitioning of the image according to criteria of 
chrominance and of apparent motion of the pixels - described 
later - is dubbed ''segmentation". 
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The operation 20 comprises the comparison of the 
luminances of the pixels of a current image 14j_ with respect to 

the luminances of the pixels of an earlier image 12 This 

comparison determines the motions or the variations of luminance 
5 of the pixels of the image 14-j_ with respect to the pixels of tne 

image 12-i_. 

A so-called ''optical flow" vector 15p representative 

of the motion or of this variation of luminance of this pixel P 
is thus allocated to each pixel P of the current image 14^. To 

10 do this, this vector 15p is described by coordinates (dx, dy) 

characterizing the motion of the image at the pixel P with 
coordinates (x,y). 

During the operation 22, a fragmentation F of the 
image 14-l into fragments F]_, F2, •••^j^ based on the colour is 
15 performed. To do this, the luminance information for the 
image 14i is used, together with the evaluations 16 and 18 of 

the apportionment of the red and blue colours via respective 
signals and V-^ for each pixel P-l of the image 14 j_. 

Thereafter, on the basis of the fragments Fi, F2, 
20 -.-F^ thus obtained, the final segmentation of the image 14;^ is 
obtained by grouping these fragments F^ into parts Rj^^ calieo 

regions, according to an operation 30 involving motion 

parameters - this operation being described later. 

There are numerous methods making it possicle to 
25 perform this fragmentation F of the image into fragment.^ F_, 

...F^ of -homogeneous colour - homogeneity being aefmed 

according to the quality of fragmentation demanded. 

Moreover, it should be noted that other criteria r.jv 
be used to perform this fragmentation. 

30 For example, the fragmentation can be f^-rrforrred 

according to a so-called ''texture" criterion basec; on the 
spatial distributions of the grey levels in the imaae. To go 
this, these textures are characterized by criteria of 
homogeneity, of contrast, of favoured orientation or of 

35 periodicity. 
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In this description, fragmentation based on the 
merging, pairwise, of neighbouring fragments of similar colour 
is used. More specifically, on the basis of two so-called 
"^starting" neighbouring fragments, a new so-called ''finishing" 
5 fragment comprising the two merged fragments is created. 

Thereafter, this method is repeated by considering 

this finishing fragment to be a new starting fragment. 

To determine the sequence of merges - a single merge 
being performed at each step - we calculate a cost Cf^ 

10 associated with each envisageable merge. 
H This cost Cfu, the calculation of which is described 

p? hereinbelow, is representative of the difference in colour 

§1 between the two fragments whose merge is envisaged. 

^2 Thus, by merging (F-^uFj) the fragments and Fj whose 

m .15 cost Cf^ of merging is the lowest among all the envisaged costs 

or merging, we merge the fragments which are most similar 

p chrominancewise among all the fragments which may be merged. 

riJ In this embodiment, the calculation of the cost Cfy of 

h:{ merging between two fragments Fi and Fj is as follows: 

Q 20 

m NjN: 

^ Ni + N j [(^i-^j^^ ^ (Ui-Uj)2 + (Vi-Vj)2] 

In tnis formula, N^^ is the number of pixels m the 

fragment F-[ and N-^ is the number of pixels in the fragm.ent. F-j 
25 and (Y-L-Yj), (Ui-Uj) and {V3_-Vj) represent, respectively, the 

differences in luminance and in colours between the two 
fragments F-;^ and Fj . 

On the basis of this merge oetween the two fragments 
F-i_ and F- ^ a new fragment Fj.^ = ^i^f^j comprising the pixels of 
30 the two starting fragments F-^ and Fj is obtained. This new 
fragment Fj^^ therefore comprises Nj^ = N-^ + Nj pixels. 

This new fragment F]^= f'i'^F'j is then characterized by a 
luminance Yy^ equal to the mean of the luminances of the merged 
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fragments^ weighted by the number of pixels present in each 
fragment . 

More precisely, when merging the fragment Fj^ with the 
fragment Fj, the new mean luminance of the fragment F]^ is equal 

5 to: 

Yk - (Ni*Yi +Nj*Yj)/(Ni+Nj) 

Likewise, we define the parameters U]^ and V]^ of colour 
differences of the new fragment F]^ as, respectively: 
Uk={Ni*Ui +Nj*Uj)/(Ni+Nj), 
10 Vk= (Ni*Vi +Nj*Vj)/(Ni+Nj) . 

□ Each pixel constitutes a starting fragment for the 

;rf first fragmentation step. Subsequently, the unmerged pixels 

\l remain fragments. 

in However, a minimum number N of fragments is specified 

S: 15 so as to stop the segmentation when this number of fragm.ents is 

s reached. 

Q Thus, we obtain N fragments F^, F2, ... F^ making up 

p the image 14 j_, each of these fragments comprising a given number 

111 of pixels N]_, N2, ...N]vQ. 

20 The cost of the merge between Fj_ and Fj being 

proportional to N-L.Nj/N-j_+Nj , if we assume an isotropic 

commencement of merging, then the bigger the number of pixels 

concerned in a merge, the higher will be the cost of this merge, 

thus favouring the merging of small fragments and hence an 

25 isotropic fragmentation of the image. 

Thereafter, in the course of an operation 24, a 

parametric model characteristic of the motion of all the pixels 
of each fragment F^, F2, Fj^ as previously obtained is 

estimated. 

30 To do this, each fragment Fj_ is characterized by a 

parametric model 24 3_ of motion linked to tne horizontal 
component dx and vertical component dy of the motion vector 15p 
of each pixel with spatial coordinates {x,y) of the fragment F^^. 

More precisely, an affine model 24 with 6 parameters 
35 {a,b, c,d,e, f ) is chosen, such that the components dx and dy of 
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the motion vector 15p of a pixel P with coordinates (x,y) are 
equal to 

dx=a+b.x+c.y, dy=d+e.x+f.y. 

Thus, a single model 24 of motion parameters with 6 

5 components describes the motion of all the pixels of the 
fragment considered. 

The parameters a, b, c, d, e and f of the model 24j_ 

are determined according to the so-called least squares 
technique from the motion vectors 15p estimated at each pixel 

10 during step 20. 

More specifically, on the basis of a starting model 
Q 24 'i (a',b',c*,d',e',fMf its parameters are made to vary in 

such a way as to minimize a deviation E^q between the ''real" 
vectors 15p and the vectors calculated from this model according 

15 to the above formulae. 

To evaluate this deviation Ej^q, we calculate the sum 
of the squares of the differences between the motion vector 15p 

of each pixel and the vectors reconstructed from the model 
described previously. 



20 For example, for a vector Ib^ with coordinates (dx. 



HI 

Pi 

dy) , modelled by a model 24 -[ (a,b, c, d, e, f ) : 

Erno=(dx - (a + b.x + c.y))2 + (dy - (d + e.x + f.y))2. 

The final parametric model (a, b, c, d, e, f ) is obtained 

when this deviation or ''variance'' Ej^q is minimal. This variance 

25 in the modelling of the motion of the fragment is then 
designated as Var24i. 

Generally, the evaluations described here take no 

account of so-called "outlying'' values, that is to say of values 

which differ excessively from the globally estimated values. 

30 In parallel, during an operation 26, the f ragm.entation 

F can be comoared with the segmentation of the previous image 
12i. 

For example, any correspondences 28 between the 
location of fragments Fj_ and Fj and the location of a region 
35 R'j_, are identified, this region R'^ being the final result of 
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the segmentation of the image 12j_ according to an operation 30 

described later. 

These correspondences 28 may be used during operations 
requiring the tracking of object (s) over a series of images, for 
5 example in the road traffic field. 

Thereafter, during the operation 30, a method of 
grouping similar to the method of merging described during the 
operation 22 is performed. Thus, an iterative process is applied 
involving two neighbouring fragments which minimize a grouping 
10 cost C^^ef this grouping creating a new starting fragment. 

During this operation 30, the cost of merging is 
evaluated from the models 24^ of motion parameters of each 

fragment. 

Thus, the two fragments grouped together at each step 
15 of the operation 30 are the fragments exhibiting the greatest 
similarity of motion among all the pairs of neighbouring 
fragments - 

For example, by considering two fragments Fj^ and Fj 
characterized by respective parametric models 24^ (a^, d^, c^, 
20 d-i_, ej_, fj^) and 24j (aj, bj, Cj, dj, ej, fj), the similarity of 

motion between these two fragments is calculated as follows, 
where it is assumed that the fragment F-j_ is of larger size than 

the fragment Fj : 

A motion vector 15pj/j[ is calculated for each pixel of 
25 the fragment Fj according to the parametric model 24;^ ror the 
fragment F-j_. Thus, for a pixel with coordinates (xj, y^ ' ot F j , 
we calculate the vector 15pj/j_ with coordinates (dxj/^, dyj/^) 

according to the following formulae: 
dxj/i = ai ^ bi Xj + Ci yj 
30 dyj/i = di+ eiXj + fiyj 

Thereafter, the motion vector l^pj/j of this pixel is 
evaluated according to the parametric model 24 -j ror this 

fragment, rhat is to say 

dxj/j = ^j ^jXj yj 
35 dyj/j = dj + ejXj + fjyj 
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Finally, the difference between these two vectors 
ISpj/i and 15pj/j is evaluated by calculating a difference Apj/-^ 

Apj/i = (dxj/j - dxj/i)2 + (dyj/j - dyj/i)2. 

The mean of the Apj/j_ of all the pixels of Fj is then 
5 calculated so as to obtain an evaluation Aj/^ of the difference 
between the parametric models of the two fragments F3_ and F^ . 

Subsequently, the fragments Fj_ and Fj whose difference 
Aj/i of motion is less than a predetermined threshold - are 

grouped together - this threshold being all the smaller the 

10 greater the agreement between fragments has to be in order for 

these fragments to be grouped together. 

However, during this grouping operation 30, no new 

motion parameter 24 is calculated in respect of a fragment 

created by a grouping. This is because these complex 

15 calculations would require overly large durations. 

This is why, during this operation 30, a motion vector 

equal to one of the motion vectors of the two grouped fragrrtents 

is allocated to each fragment created by a grouping. 

In this embodiment, the motion parameter of the 

20 grouped fragment of smallest size is allocated to the fragment 

resulting from the grouping. 

For example, we consider the grouping between two 
fragments F*j_ and F'j such that the number of pixels N'-, of the 

fragment F'-^ is greater than the number N'j of pixels ci the 

25 fragment F*j. The calculation is speeded up by allcr.::t ir.- 

motion vector 24']^ equal to the vector 24 *i to the fragment 

obtained through the grouping of F'i and of F'j. 

These iterative groupings are performed ur.ti. 

specified number of fragments is obtained. 

30 When this grouping operation is completed, a given 

number of ''final" fragments or of regions Rj_ which characterize 

the segmentation of the image according to this methcd i5 tnen 
obtained. 

The set of pixels P included in a region is tnen 
35 homogeneous in terms of motion, this parameter being 



characterized by a unique model 24i for the entire set of pixels 
of the region R^. 

Before transmitting (or recording) this segirientaticn, 

a marking operation 33 is then performed, in the course of which 
the regions making up the image 14j_ are identified. Thus, when 

the image 16-i_ posterior to the image 14-l is analysed, iz will oe 

possible to use this segmentation to undertake the operation 26 
with the image 16j_. 

To do this, a last step 35 is required in the course 

of which this segmentation 35 is assigned a delay corresponding 
to the lag in the appearance of the next image 16-i_. 

As mentioned previously, the fragmentation F of the 
image into fragments F-l is stopped according to a number-of- 

f ragments-obtained criterion. Thus, when the number of fragments 
obtained reaches a certain threshold, the merging method stops. 

The present invention results from the finding that 
this stoppage criterion does not yield an optimized result. 
Specifically, according to this method, the fragmentation of a 
very simple image produces an excessive fragmentation of the 
latter while the fragmentation of a complex image - comprising 
multiple details - produces an inadequate fragmentation. 

Summary of the invention 

The present invention makes it possible to obtain a 
fragmentation which adapts to the nature of the images. 

It relates to a method for the fragmentation of images 
using an evaluation of the cost of merging between fragments, 
this cost of merging being of the form cost (F-jUF-i . = A *-E, 

where A is a term proportional to a colour difference and/ or 
luminance difference and B is a term inversely prcporti :r.al tc a 
number of pixels, and this method is characterized in that the 
cost is weighted by a new coefficient C consistent with a number 
of pixels. Since the cost of a merge can then be evaluated with 
respect to a colour difference or luminance difference, it is 
possible to stop a fragmentation when a colour threshold and/or 
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luminance threshold is reached, thus tailoring the fragmentation 
to the complexity of the image. 

In one embodiment, the coefficient C is proportional 
to 2/ Nj^ where Nj^^ represents a mean size of the fragments Fj_, 

such as a mean number of pixels per fragment Fj_. 

The selection parameter (s) is (are), for example, at 

least one of the following parameters: the luminance, the 

chrominance and the texture. 

According to one embodiment, each iterative merge 
relates only to two neighbouring fragments and F^ . 

Moreover, the factor A is, in one example, 

proportional to , where and are representative of 

Ni + Nj 

the size of each merged neighbouring fragment Fj_ and F^ ~ it 

being possible for the size of a fragment to be the number of 
pixels included in this fragment. 

According to one embodiment, the factor B is 
proportional to [(Yi-Y^)^ + (U^-U^)^ + (Vi-Vj)^], where (Yi-Yj), 
(Ui-U2) and (V^-Vj) represent, respectively, the difference 

between the luminances and the blue and red colour signals of 
each of these two fragments. 

Moreover, the invention also relates to a method of 
grouping the fragments of an image which are obtained by a 
method according to one of the previous embodiments. This method 
of grouping uses a model of motion individual to each fragment 
Fj_, this model of motion being determined with a Known error or 

variance so as to allocate to the fragment created by this 
grouping one and the same model of motion. 

Thus at each step of this method the two fragmtents Fj_ 
and Fj which minimize a grouping cost C^^e proportional to an 

evaluation of the difference between the models of motion of the 
two fragments Fj_ and Fj are thus grouped together. 

In one embodiment, the grouping cost (C^q) is 

inversely proportional to a threshold S-^q of motion such that, 

in the absence of any other stoppage test, the grouping is not 
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allowed if the evaluation Ai/j is greater than this threshold 



Sre- 



More precisely, according to one embodiment the 
grouping cost C^^^ calculated according to the formula: 



NixNj j- . 1 
^re - xjj+Ni ^i/j)/^reJ' 



Ni+Nj 

where N-l is the number of pixels of the fragment and Nj is 
the number of pixels of the fragment Fj . 

Moreover, in the case where the fragment Fj is of a 
greater size than the size of the fragment F^, the evaluation 
Aj/i of the difference of motion between these fragments Fj_ and 
Fj comprises, for example, the following steps: 

- calculation of a motion vector for each pixel P 
(x,y) of the fragment Fj according to the model with parameters 
(a-L, b-L, cj_, d-i_, e^, f^) of motion of the fragment F-^, in such a 
way that, for a pixel P with coordinates (xj, y j ) of Fj , we 
calculate a vector 15pj/i with coordinates (dxj/j_, dyj/j_) 

according to the following formulae: 
dxj/i = ai + bi Xj + Ci yj 
dyj/i = di+ e^xj + fiyj 

- evaluation of the motion vector l^pj/j of this pixel 
P according to the model 24 j with parameters (a^, bj_, c^, d^, 
ei, fi) of this fragment Fj, that is to say 

dxj/j = a-^ + bjXj + Cj yj and 
dyj/j = d3 ^ ejXj + fjyj 
. - evaluation of the difference Apj/j^ between these two 
vectors i^pj/i and i5pj/j by calculating 

Apj/i = (dxj/j - dxj/i)2 + (dyj/j - dyj/i)2 

- evaluation Aj/j_ of the difference in motion between 
fragments Fj_ and Fj by summing the Ap^/^ values ci c:ll che 
pixels of the fragment Fj and then by dividing by N j . 
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In one embodiment of this method of grouping the size 

of a fragment is defined as the number of pixels included in 

this fragment - 

Moreover, the grouping threshold S^^^ is, preferably, 

5 proportional to an error or variance of the modelling of motion 
of the fragment F^. 

According to one model of embodiment, the grouping 
threshold Sj^^ is proportional to the maximum between the 

variance of the model of motion of the fragment Fj_ and the mean 

1^1, 10 of the variances of the models of motion of fragments of the 

Q image . 

y In one embodiment, the grouping threshold Sr-^ is 

%J higher when fragments whose pixels were included in one and the 

i*! same region of a previous image are grouped together. The 

flj 15 grouping is thus less selective in this case. 

However, a luminance prediction test can be used 
before applying a less selective grouping threshold Sj-e- 
y Thus, according to one embodiment, the fragment Fj 

£j being of a greater size than the size of the fragment F^, the 

flj 20 luminance prediction is performed by calculating, for each pixel 

P with coordinates (xj, y j ) of the fragment Fj, the vector with 

coordinates (dxj/-^, dyj/j_), the luminance predicted at P (xj, 

yj) in the current image being that found at (xj - dxj/^, 

yj - dyj/i) m the previous image. 

2 5 Moreover, in order to encourage the groupma of 

fragments into regions of arbitrary size Nj^-^j^, it is possible to 
use a grouping stoppage test using the grouping cos: Cr-e 
multiplier by a coefficient such that, calling N't^ the maximum 
between tne number of pixels of the fragment Fj_ and Nrnln^ 

30 N'j the maximum between the number Nj of pixels of the fragment 
Fj and l^^^ir^.- this case, the stoppage test may be written: 
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However, it is possible to use another grouping 

stoppage criterion such as 

Cre{FiuFj) > 1 

so as not to discourage the grouping of fragments of size 
greater than ^min- 

Other characteristics and advantages of the invention 
will become apparent with the description of certain of its 
embodiments. 

Detailed description of preferred embodiments 
In this description of a preferred embodiment of the 
invention, use is made of a calculation of the fragment merging 
cost similar to that described above during the operation 22, 
that is to say : 

- Cost (FiuFj) = [(Yi-Yj)^ + (Ui-U-j)^ + (V^-Vn)^] 

Ni + Nj 

It is found that, in this formula, the cost of n^.ergma 
the two fragments and Fj is equal to the product of factors A 

and B such that: 

^ = ^ = [(Yi-Yj)^ + (Ui-U^)^ + (Vi-V- n 



Subsequently, in order to use a stoppage rritericr. 
consistent with a colour difference and luminance G^:i-rre:.jv, 
two additional steps are introduced: 

Initially, the cost of any envisaged r^^-ne i-r' 
multiplied by a factor C consistent with the inverse o: i nurr^er 
of pixels. 

In a preferred embodiment, C = 2/ Nj^, -sT.ere i:^. 
represents the mean size in pixels of the fragm.ents F-. at a 

given step of the fragmentation. Stated otherwise^ if .r. a given 
step of the fragmentation we consider k fragments F^, 7 , ...Fv; 

respectively comprising N^, ... and Nj< pixels; 

% = (Ni + N2 + . . - + Nk)/K. 
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It is understood that the factor C is obtained by 
considering that 

Ni = Nj = Njj^, from which it follows that 

and to obtain AC = 1, we must have C=-f^ . 

Nm 

Subsequently, the product cost (FiuFj)*C, that is to 
say A^B^C, is compared with a fixed threshold Sq. 

If A*B*C < S^/ the deviation in terms of luminance and 

colours between the two fragments is less than the threshold 
deviation fixed for stopping the fragmentation. The merge is 
therefore permitted since the two fragments are sufficiently 
similar. 

If A*B*C > Sq, the deviation in terms of luminance and 

colours Joetween the two fragments is greater than the fixed 
threshold, the merge is not permitted and the fragmentation is 
stopped. 

Hence, the test for stopping the fragmentation no 
longer corresponds to a specified number of fragments to be 
reached, but to an inter-fragment homogeneity threshold Deyond 
which the merges no longer take place. 

Thus, as a function of the complexity of the image, 

the number of regions resulting from the segmentation of the 

image is variable. 

Moreover, by using the term which is independent 

of the fragm.ents considered in the merge, the merging ot small 

fragments is encouraged. 

Specifically, the cost of the merge is proportional to 

the aifference in luminance and colours - factor B - and to a 
factor A"C equal to (N^ x N j /N-j_+Nj ) . 

Owing to this factor A*C, if the fragments Fj_ and Fj 
have a size less than the mean, this factor is less than 1. 
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Conversely, if the fragments F-^ and Fj have a size 
greater than the mean, this coefficient A*C is greater than 1 

and the cost of the merge is increased. 

The threshold of merging is consistent with tne 

square of a luminance difference and/or colour difference. It 
has been found that a threshold of 3*a2, where A represents 6 
grey levels on each component, yields satisfactory results. 

According to another aspect, the present invention 
also results from the finding that the grouping of fragments F^ 

during the operation 30 according to a fixed motion difference 
criterion does not optimize this merging operation. 
Specifically, the use of a fixed threshold does not allow good 
stability of segmentation. 

In fact, when processing an image, the motion 
estimates and their accuracy vary considerably between 
fragments. For example, a first fragment can be processed with a 
motion estimation varying between - 1 and + 1 pixel, with an 
accuracy of ± 0.25 pixels, and a second fragment with a motion 
estimate varying between - 30 and + 30 pixels, with an accuracy 
of ± 2 pixels. 

It then appears that the use of a fixed threshold, for 
example for these two estimates, does not optimize the 
characterization of their motion, since this threshold would be 
unsuitable - too limited and/or too wide - for these fragments. 

This is why, in one embodiment, which can be used 
independently of the other embodiments, a grouping cost C^q is 

calculated which can vary as a function of the fragments 
concerned and takes account of the evaluation Aj/^ of the 

difference between the parametric models of the grouped 
fragments and a grouping threshold S^^e related to the variance 
Var24i of the model of motion of the smallest grouped fragment, 
as defined previously, and to the prior segmentation of an 
earlier image. 

Subsequently, the possible groupings between fragments 
Fi and F^ are ranked according to a cost C^e^^^^^i) incurred by 
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their grouping. In this preferred embodiment, this cost 
Cre(Fi'^Fj) is calculated according to the following formula: 

Thus, it is possible to group together the fragments 
whose grouping cost is least, that is to say the fragments which 
are most similar according to the motion criterion, whilst 
imposing a grouping stoppage threshold S^e relating to the 

difference of motion between the fragments. 

The segmentation of the image is then improved since 
it is dependent on the nature of the image and not on the 
predetermined number of segments to be reached. 

The grouping threshold S^e takes account of the fact 

that, as described previously, the determination of the 
15 parametric model 24j of motion of a fragment Fj makes it 
possible to determine the variance Var24j of this model. 
Subsequently, this variance represents a margin of tolerance 
which has to be taken into account when evaluating the 
similarity of motion between fragments. 
20 Thus, in one embodiment which can be used 

independently of the other embodiments, the grouping threshold 
Sre is proportional to the variance Var24j, this threshold S^e 
being all the higher the larger the variance Var24j. Stated 

otherwise, the greater the uncertainty in "the modelling of the 
25 motion of a fragment, the greater the threshold S^e of tolerance 

for the grouping. 

• Thus, when the determination of a parametric model 24 ^ 

of a fragment F-^ causes a large margin of error - Var24j - the 

grouping threshold S^e for this fragment F^ takes this 
30 approximation into account and is higher, allowing a grouping of 

this fragment with a lesser demand regarding the similarity of 

motion. 

Nevertheless, it is possible for the grouping 
threshold to be very low subsequent to a good modelling of the 
35 motion of the fragment Fj, that is to say when Varj is low. 
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In this case, the grouping of the fragment Fj with 
other fragments is penalized if the grouping threshold is 
directly proportional to the variance of its model of motion, in 
which case the threshold would be low. 
5 To avoid this situation, in one embodiment which can 

be used independently of the other embodiments, at each grouping 
step, an estimate of the mean Var^^ of the variances Var2^^ of 
modelling the motion of each fragment Fj is obtained and tne 
grouping threshold for a fragment Fj is fixed at the maxim^om Max 
10 (Var24j, Var^^) between the variance Var24j individual zo the 
H grouped fragment and the mean variance Var^^. 

g Empirically, the better results are obtained for a 

§1 grouping threshold fixed at O.S"^ Max(Var24j, Varj^) . 

in More specifically, we can consider the grouping 

ffl 15 between two fragments Fi and Fj, these two fragments oemg tne 

fragments which minimize the grouping cost C^^ calculated for 
p all the pairs of starting fragments. 

Jj By considering the fragment F^ to be smaller than the 

S fragment Fj, for example according to a number-of-pixels 

y 20 criterion, the grouping threshold Sj^q is evaluated with a 

tolerance dependent on the variance Var24i. 

However, in parallel, the mean variance Var^p^ of ail 

the starting fragments is evaluated. Thus, if during this step 
there are k fragments F^, F2, ...F]^ of respective variance 
25 Var24i, Var242f •••Var24k, we have: 

Varm= (Var24i + Var242 + Var24k)/k 
Subsequently, if this tolerance Var^^ is hiaher than 
the variance Var24i, the threshold S^e be determined as a 

function of Var^^. Thus, the grouping of the fragment is net 

30 subjected to a lesser tolerance than in the '"mean" case. 

Moreover, this grouping threshold S^e is evaluated as 

a function of the prior segmentation of an earlier imaae. 

Specifically, this embodiment, which can be used 
independently, results from the finding that the operation 26 of 
35 comparison between the fragments Fj_ emanating from the colour 
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fragmentation 22 and the regions R'^ emanating from the previous 
segmentation of the earlier image 12j_ makes it possible to 

simplify the calculations by making predictions with the aid of 
any correspondences between the segmentation of an image 12-^ and 

5 the fragmentation of the following image. 

Specifically, by virtue of this comparison 26, it is 
possible to determine whether fragments Fj_ and Fj of the image 

undergoing processing correspond to zones of the image wnich 
were previously grouped together in the previous image, that is 
10 to say to one and the same region. 

This is why, in an embodiment which can be used 
0 independently of the other embodiments, assuming that these 

J: fragments are grouped together again, a higher grouping 

Iji threshold is then fixed in respect of these fragments, thus 

p 15 encouraging their grouping. 

For example, this threshold S^^ can be fixed m the 
Q following manner: 3^^= 4 * Max (Varj^, Varj), 

S • However, it is possible for this assumption to be 

ill incorrect, for example when a displacement of objects occurs 

y 20 between two images. 

In this case, too high a grouping threshola can 

encourage a grouping of fragments which do not correspcna to 

objects with the same motion. 

To avoid this situation, a condition regardir..T the 
25 variation of luminance of the fragment Fj from the previous 
image 12 -j is fixed, making it possible to detect any object 
motion between the two images. 

For example, this condition can be fixed by 5:;a.ysinq 
the luminance predictions . 

30 This luminance prediction, for example associated with 

the fragiTient Fj and with the parametric model 24 1, is defined m 

the following manner: 

At each pixel P with coordinates (xj, y j ; cl the 
fragment Fj, the vector 15pj/i with coordinates (dxj/^, dy./i) 
35 is calculated as explained previously. Subsequently, the 
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predicted luminance at P (xj, yj) in the image 14^ is thai: found 
at (xj - dxj/i, yj - dyj/^) in the image 12 j_. 

A prediction error is then defined as being the mean 
of the differences between the predicted luminances and the real 
5 luminances of all the pixels P with coordinates (xj, y j ) of tne 
fragment Fj . 

Such a luminance prediction error is calculated over a 
relevant fragment before the grouping thereof and after the 
grouping thereof with another fragment. The merge is accepted if 
10 this prediction error does not vary excessively, the allowed 
error being dependent on the predetermined resolution. 

In this case, it can be assumed that there is 
correspondence between the fragments F-^ and and a higher 

grouping threshold can be applied. 
15 Empirically, the error between these predictions is 

fixed at a deviation of seven grey levels. 

Taking the above remarks into account, it is apparent 
that the use of a stoppage criterion according to a calculation 
of grouping cost C^e as defined by the formula: 

20 C^e(FiuFj) = ^ [( Ai/j)/S^e] 

makes it possible to take account of characteristics 
individual to each fragment. 

Additionally, it is possible to rank the possible 
groupings according to the cost Cj^e and to encourage the 

25 grouping of the fragments which are most similar acc .rJmg to 
this criterion. 

• Moreover, it is possible that the user c: such a 
segmentation algorithm may desire preferably to obtain regions 
of a given size, for example comprising a number l<lrnin of pixels. 

30 In fact, in an embodiment which can be used 

independently of the other embodiments, after eacn grouping of 
fragments, a stoppage test which takes this choice into account 
is applied. For example, by calling i_ the maximum between the 
number Nj_ of pixels of the fragment Fj_ and N^^^^n j ^he 
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maximum between the number Nj of pixels of the fragment Fj and 
%in^ impose the condition: 

C^e(FiuF:) - > 1 

5 

N'i+N'i 

Subsequently, the term ^^^.j" encourages the grouping 

of the fragments of size less than Nj^-l^. 

However, this term disadvantages the grouping of 
fragments of large size. 
□ 10 This is why, in one embodiment, a second grouping 

Q • stoppage criterion is used alternately with the grouping 

stoppage criterion described previously. More precisely, a 
second grouping cost C';^e is considered, such that: 

J ^'re = [( Ai/j)/S^e] • 

■5 1^ Thus, by using this second grouping cost, it is 

y possible to devise a second stoppage criterion which allows 

Q fragments whose difference in motion is less than the grouping 

in threshold to group together. 

For example, C^e = [( ^i/j)/Sre] > 1 is imposed as 
20 stoppage criterion. 

In this embodiment, the use of the two stoppage 
criteria is alternated in such a way as to obtain a segmentation 
of the image which does not ''penalize'' a fragment for its size. 

2 5 What is claimed 

1. Method for the fragmentation (F) of images (14j_) 

into homogeneous regions (R^), this fragmentation (F) using 
iterative merges of fragments F^ and Fj which are as similar as 
possible according to at least one selection parameter, rhis 
similarity being evaluated by a product A*B of two factors A and 
B, A being consistent with a number of pixels and B being 
consistent with the selection parameter (s) , characterized in 
that a merge is performed when the product A^B^C is less than a 
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